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ABSTRACT 


This study investigated whether elementary students could collectively advance a knowledge-building discourse through 
judgments of promisingness and epistemic reflection. Supported by the Promising Idea Tool and its accompanying 
pedagogical design, 32 grade six students were involved in judging the promisingness of their community ideas, refining 
ideas iteratively, and explicitly reflecting on their knowledge-building discourse using knowledge-building principles. 
We analyzed students’ online discourse using multivariate methods, including socio-semantic network analysis and 
content analysis. Socio-semantic network analysis indicated that the students progressively advanced their community 
ideas and the community became more connected over time. Content-analysis results indicated that sixth graders could 
collectively improve their discourse by contributing diverse ideas, negotiating a fit, and generating questions, using 
problem-centered uptake moves and synthesis notes. The study’s findings have important implications for the design of 
technology-rich environments, and shed light on how teachers can use them to help learners engage in productive 
collaborative inquiries. 
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1. INTRODUCTION 


Helping students develop higher-order skills such as knowing how to inquire and collaborate, cultivate 
agency and metacognition, and create knowledge is an educational mission. Preparing students for 
collaborative, sustained and creative scientific practices that advance their higher-order competencies is 
therefore essential (Bransford et al., 1999; Zhang et al., 2018). Collaborative knowledge building, an 
influential pedagogical model in learning sciences, shows great potential for helping students develop these 
higher-order competencies. 

Collaborative knowledge building, pioneered by Scardamalia and Bereiter (Scardamalia & Bereiter, 2014) 
emphasizes collective responsibility, improvable ideas, student agency and metacognition, and creative and 
promising ideas. In knowledge building, students’ work is supported by Knowledge Forum, an online 
platform that facilities their knowledge building. The knowledge-building process is so emergent and 
complex that students, as active agents, need to continually identify promising ideas and evaluate the 
promisingness of ideas (Chen et al., 2015). To meet their emergent goals and invent new goals (Yang et al., 
2016), they must reflect on the process, based on their assessments of the state of the learning process and its 
products (Zhang et al., 2018). The ability to judge the promisingness of community ideas and continuously 
reflect on the knowledge-building process to identify knowledge gaps and high points for further inquiry is 
critical to productive knowledge building. However, most Chinese elementary school students tend to receive 
teachers’ instructions rather than participate as active players in learning. They generally lack the skills to 
judge the promisingness of ideas or engage in metacognitive reflection. 

To address this challenge, this study designed a knowledge-building environment, augmented by 
judgments of promising ideas and epistemic reflection. To help students engage in productive judgments of 
promising ideas, students were provided with the Promising Ideas Tool, an analytic tool developed by Chen 
et al. (2015) and now embedded in Knowledge Forum. The Promising Ideas Tool can help students select 
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promising ideas from their community’s ideas and support collective decision making to identify promising 
directions for further inquiry (Chen et al., 2015; Chen, 2016). 

The purpose of this study was to investigate whether sixth graders could collectively improve their 
knowledge-building discourse in a knowledge-building environment augmented by judgments of the 
promisingness of ideas and epistemic reflection. The study addressed the following research questions. 
(1) Did students’ contributions and collaboration change during the knowledge-building process? (2) What 
was the nature of the knowledge-building discourse? (3) To what extent did students improve their 
discourse? 


2. METHODS 


2.1 Research Context and Participants 


This study was conducted in an elementary school in Wuhan, China, with students mostly coming from 
middle- or upper-middle-class families. Thirty-two grade six (11-12 year old) students from one class 
participated in the study. The students studied the unit “shapes & structures” for eight weeks, with three 
45-minute science lessons each week. They had no previous experience of knowledge building. The teacher 
was an expert in teaching science, with five-years of teaching experience. However, this was the first time the 
teacher had adopted a knowledge-building pedagogy. 


2.2 Pedagogical Design 


To help students engage in productive knowledge building, the teacher adopted the following components 
from the knowledge building pedagogical design proposed by van Aalst and Chan (2012): 

Component I: Creating a collaborative classroom culture and helping students develop competencies of 
inquiry-and-explanation and collaboration. To help students develop inquiry, explanation, and collaboration 
competencies, the teacher created several opportunities for student groups to engage in hands-on 
experiments, and for whole class discussions. For example, students in small groups collaborated to explore 
the effects of width and thickness on the resistance of materials to bending, built bridges using newspapers, 
and made towers using bottles and tubes (Figure 1). These activities helped them not only to learn to 
collaborate and communicate, but to develop the ability to inquire by asking questions, proposing 
hypotheses, designing and conducting experiments, and drawing conclusions through discussions and 
negotiations over the fit of diverse ideas. More important, these principle-based activities created a 
collaborative-inquiry ethos and norms for democratic participation. 

Component 2: Advancing inquiry-oriented and idea-centered discussion through the Knowledge Forum 
discourse. The students shared their questions and ideas and discussed their initial experiments and findings 
in small groups and whole-class conversations. Focusing on the questions, the student groups conducted 
research with the help of books, experiments, online resources, and materials from their teacher. Extending 
their face-to-face knowledge building talks and interactions, the students recorded their questions, 
experimental findings, and ideas in Knowledge Forum to produce an ever-deepening online discourse 
(Figure 1). Knowledge Forum provided students with epistemic scaffolds such as “My idea,” “My theory,” 
“My evidence,” “My conclusion,” and “I need to understand” to help them develop the ability to inquire and 
explain. 

Component 3: Deepening knowledge advances through judgments of promising ideas and explicit 
epistemic reflection. After the students had contributed a reasonable number of notes to Knowledge Forum, 
they were instructed to use the Promising Idea Tool. To help students engage in productive judgments of 
promising ideas, they were provided with metacognitive prompt sheets (e.g., including prompts like 
“My Analysis,” “My Problem,” “My Plan’). These helped them reflect on and monitor their discussions and 
plan the future direction of discussions. We also provided students with prompt sheets to help them 
synthesize their clusters of notes. 
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Figure 1. Knowledge Forum Views and Notes, and Hands-on Activities 


3. DATA ANALYSIS AND RESULTS 


The data sources for this study were primarily the computer notes the students posted on Knowledge Forum. 
The students contributed 247 notes in total. 


3.1 Research Question 1 


Research Question 1: Did students’ collaborations and contributions change during the knowledge-building 
process? 

To reveal the patterns of students’ collaborations and contribution, we used a socio-semantic network 
analysis tool: Knowledge Building Discourse Explorer (KBDex, Oshima, Oshima, & Matsuzawa, 2012). 
KBDex was developed to analyze knowledge-building discourse, and it could support the metrics of the three 
different networks: degree centrality, betweenness centrality, and closeness centrality. Therefore, it could be 
used for visual inspections of semantic relationships, analyses of pivotal points and phases, and trend 
analysis. We selected 85 words from students’ online discourse for analysis. The agreement between the two 
raters was 83%, and disagreements were resolved through discussion. We argued that the keywords could 
represent the discourse content, so the closeness of keywords’ links and the quantity of keywords could 
represent the density and diversity of discourse content respectively. 


Phase 3 


Figure 2 (a). Snapshots of the network of students over time 
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Figure 2 (b). Snapshots of the network of keywords over time 


Figure 2. Snapshots of the network of students and keywords over time. Notes: The circle represented the 
student/keyword, and the connection between the two circles represented the relationship between them. The 
thickness of the line between the two circles represented the degree of correlation between them. 

To investigate the changes of students’ contributions and collaborations, we first identified two pivotal 
points to divide the knowledge-building process into three phases. Figure 2(a) shows that the density between 
the students increased from Phase 1 to Phase 3. Figure 2(b) shows that the keywords network structure was 
becoming denser, and extremely different keywords appeared from Phase 1 to Phase 3. These results 
suggested that the community was becoming interactive over time, and that students were engaged in 
productive knowledge building. 

We further analyzed the patterns of contribution of students in the community through the changes to 
each student’s sum degree coefficients in the three phrases (Figure 3). The gaps in the sum degree 
coefficients between students in Phase | indicated that their individual contributions were not democratic. 
However, the gap of the sum degree coefficients among individual students became smaller in Phase 3, 
indicating that they had progressively taken collective responsibility for advancing community knowledge. 
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Figure 3. The changes of Sum Degree Coefficients of Individual Students in Three Phrases 


3.2 Research Question 2 


Research Question 2: What was the nature of the knowledge-building discourse? 

To characterize the students’ online knowledge-building discourse, we first pre-processed their notes into 
inquiry threads. This provided context for the subsequent content analysis of the notes within each inquiry 
thread. An inquiry thread was defined as a sequence of notes addressing a single problem (Zhang et al., 
2007). Two hundred and forty-seven notes were put into 12 inquiry threads through inquiry thread analysis. 
To check the coding reliability of the inquiry thread analysis, two raters independently completed the task on 
30% of the notes, resulting in an inter-rater reliability of .80 (Cohen’s kappa). 

After pre-processing the students’ notes, we conducted content analysis using the inquiry thread as the 
unit of analysis. We developed a coding framework with coding examples to code the notes in each inquiry 
thread. The development of the coding framework involved an iterative coding process of theory- and 
data-driven approaches. The coding schemes (Table 1) included four main categories and corresponding 
subcategories, and drew upon theoretical frameworks for social, cognitive and meta-cognitive processes of 
knowledge construction (Authors, 2016; van Aalst, 2009). Two raters independently coded 30% of the notes. 


The inter-rater reliability was .78 for questions, .83 for scientificness of ideas, .78 for complexity of ideas, 
and .79 for community (Cohen’s kappa). 
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We selected nine large inquiry threads and presented the numbers of questions, ideas and collective 
responsibility in them. The inquiry threads defined as large included more than 10 notes each. Table 2 shows 
that the elementary students in this class were engaged in explanation-oriented discourse: they focused more 
on explanatory than factual questions (24 compared with 7). They also generated more notes with 
elaborations than notes with simple claims (72 compared with 39). This result indicated that the students 
engaged in a deep rather than a superficial knowledge-building process. 

Table 2 also shows that students invested a lot of effort in collectively advancing their community ideas. 
For example, they contributed many notes to negotiating a fit between diverse ideas (65 notes), showing idea 
uptake (41 notes), and rising above the community’s ideas (17 notes). These results indicated that students in 


this class could take collective responsibility to improve their ideas progressively. 

Overall, the above results suggested that elementary students in this class were able to assume high-level 
responsibility to collectively accomplish a knowledge-building discourse. They engaged in productive 
collaborations and gradually improved ideas in the communal space. 


Table 1. Coding framework for content analysis of students’ online discourse 


Categories Subcategories Definitions Examples 
Question Fact-seeking Questions can be answered by factual Please discuss those shapes such as ”—”,”W”, 
information 
“ET,” L” which impacts the material most? 
Explanation- Open-ended question that can only be Why shape "W" is the the most resistant to bending? 
seeking answered by elaborative explanations 
Scientificness Naive Absolutely incorrect conceptions or theories, | The combination of arch, sphere, and frame structure 
of ideas or naive ideas. don’t have any drawbacks. 

Hybrid Ideas are basically wrong, but the I think shape "W" has a greater impact. Because 
understanding and explanation of the ideas shape "W" paper is wider so it’s more durable than 
are scientific under certain restrictions. other shapes. 

Basically The understanding and explanation of the I think the impact of shape "W" is even greater. 

scientific concepts and theories in the discussion Because "Thickness has a greater influence on the 
topics and issues are basically correct, but bending resistance of the beam." The thickness of 
are not comprehensive and precise enough. shape "W" is thicker than other shapes. So the 

influence of shape "W" is greater. 
Scientific There is clear, scientific, and comprehensive Jn response to the question “Shape Wis not the 
understanding and elaboration of the thickest, why is the strongest resistance to bending?” 
concepts and theories of the discussion One student responses that “because of the 
topics and issues, and the scientific theories triangular structure, this is the most stable form”. 
contained in the discussion can be applied to 
practice. 
Epidemic Unelaborated A description of Shape “M” has the strongest resistance to bending. 
complexity of facts terms/phenomena/experiences/simple 
ideas judgments of a problem/idea without 

elaboration. 

Elaborated Elaboration of The thickness has a greater influence on the 

facts terms/phenomena/experiences in detail. resistance to bending of beams. Like the ruler we use, 

it is not easy to bend vertically. 

Unelaborated Mention reasons, relationships, and I have done experiments today. When the material is 

explanations mechanisms without elaboration. flat, the gravity coincides. When the material is 

folded, gravity is dispersed. So the resistance to 
bending is good in “U”,”T”,”O” and other shapes. 

Elaborated Provide sufficient theories and evidences to The shape of the tower plays a vital role. In our lives, 

explanations elaborate reasons/ relationships/mechanisms. _ we can see that the towers are generally small and 

light on the top while large and heavy on the bottom 
as a triangle. This is made to make the tower's 
chassis solid and firm, preventing the tower from 
falling. To make a triangle is because the tower is 
stable on the ground. 
Collective Creating Creating elaboration or explanations of I think the resistance of bending of material is related 
responsibility shared concepts or ideas to address the central to material, thickness and width. 
understanding —_ problem; reformulating problems or focus. 


Negotiating a 


fit 


Constructing arguments or explanations in 
favor of; challenging the ideas brought in. 


Although there is a business certificate, if there is no 
application, the "W" beam is still useless. Only the 
"W" beam will be used on the construction. 
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Therefore, there are various problems in the beam. 
The beam of "W" is not only difficult to do, but also 


unstable. 
Problem- Building up on peers’ ideas to provide lL agree with your opinion. Because shape "W" is 
centred idea explanations in addressing the central really difficult to do, and it is not easy to fix it on the 
uptake problem and to deepen the inquiry including —_ ceiling. However, I have to add that the "W" shape of 
co-elaborating. the beam is not beautiful, and the home will be a bit 
strange. But what shape can the beam make? 
Synthesizing Summarizing ideas from multiple notes by The arch can be placed at the bottom because it can 
notes creating hyperlinks to a small number of withstand the weight of the dispersing force; the 
notes relevant to it, extending the referenced — sphere can be placed on top because it cannot be 
ideas and introducing a new level of placed underneath; the frame structure is played both 
conceptualization. in the building of the bridge and in the building of the 


tower. In summary, these three can be put together. 
The drawbacks are still there. The sphere does not 
play much role and can only play an aesthetic role if 
used improperly, because it will waste building 
materials. 


Table 2. Number of different categories of questions, epistemic complexity of ideas and collective responsibility in 
inquiry threads 


Epidemic complexity of 


Question ideas Collective responsibility 
n ~ eo} 
B mo} tan} o 
g 2 5 
op ' no) S Oe ey =| a 3 =| oo 
2 2 § EES 32 2 325 
g 2 = ow 8 # $3 #8 ms & es 2 
Ss £ 85 S82 £8 32 28 $5 & 32 8% 
5 5 2 BS 25 8 262 86 Orel a> ene Be 
n Z me Wes DS OH 5s WSs O05 424 ASR we 
Total notes 
(9 inquiry 32 197 7 24 39 49 27 23 35 65 41 17 
thread) 
Mean 15.67 24.33 0.89 2.89 5.22 7.56 3.56 2.89 4.56 9.78 5.11 2.22 
SD 4.92 9.81 2.00 1.61 5.04 5.34 2.13 2.61 5.00 3.93 3.18 1.30 
#1 17 21 1 4 10 3 6 3 9 8 3 4 
#2 13 20 6 2 17 8 2 4 15 10 11 2 
#3 13 18 0 3 0 2 2 6 0 4 4 4 
#4 26 42 0 4 2 4 1 1 1 3 1 
#5 12 16 0 2 3 18 3 1 2 17 5 1 
#7 11 20 0 3 1 9 6 6 1 14 6 2; 
#8 13 25 0 6 7 5 3 5 6 9 8 2 
#9 21 40 1 1 6 12 1 3 6 12 2 3 
#10 15 17 0 1 3 6 4 0 0 11 3 0 


Notes: Inquiry threads defined as large included at least ten notes each. #1—Shape and its resistance to bending, 
#2—Beam and its resistance to bending, #3—The secret of arch bearing, #44—W shape and its resistance to bending, 
#5—Beam and W shape, #6—Overview of bridge and frame structure, #7—The structure of tower and stability, 
#8—Combination of arch, sphere and frame structure, #49—The placement of beam, #10—Column and W shape. Note: 
#6, #11 and #12 are not included because they were not large inquiry threads. 
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3.3 Research Question 3 


Research Question 3: To what extent did the students improve their discourse? 

To investigate the extent to which the students advanced their knowledge-building discourse, we first 
presented discourse advancement for each selected inquiry thread, followed by a demonstration of the 
characteristics of discourses before (Stage 1) and after (Stage 2) promisingness judgments and epistemic 
reflection. 


3.3.1 Idea Improvement within Inquiry Threads 


We rated students’ personal ideas on a continuum from naive to scientific understanding, and from 
unelaborated facts to elaborated explanations, respectively, coding their personal ideas at different levels to 
distinguish the depth and epistemic complexity of them. 

We first sequenced students’ notes in the large inquiry threads based on the time of the last modification. 
Then we divided the notes into two stages (Stage | and 2, before and after judgments of promisingness and 
epistemic reflection). Finally, we compared the mean value of scientificness and the complexity of all ideas 
in the two stages. Table 3 shows that the scientificness and complexity of ideas improved from 2.95 to 3.25 
and from 2.02 to 2.43, respectively. The results indicated that the students progressively generated ideas 
related to scientificness and they elaborated their ideas with increasing epistemic complexity. 


3.3.2 Changes to Questioning, Ideation and Collective Knowledge Building 


The characteristics of the students’ knowledge-building discourse in the two stages were compared with the 
aggregated results for each stage (Table 3). Table 3 shows that there were differences between the two phases 
for questions, ideas and community knowledge. The students’ notes in the later stage showed greater 
explanatory power, and focused much more on theory building (e.g., problem-centered idea uptake) and the 
review and ‘rise-above’ of ideas. 


Table 3. Changes of Question, Idea and Collective Responsibility Over Time 


Stage | Stage 2 
Total notes 108 89 
Frequency _ Percentage Frequency _ Percentage 
Question Fact-seeking 8 TAI 0 0.00 
Explanation -seeking 15 13.89 9 10.11 
Scientificness of ideas Naive 2 1.85 0 0.00 
Hybrid 15 13.89 8 8.99 
Basically scientific 42 38.89 39 43.82 
Scientific 15 13.89 26 29.21 
Epidemic complexity of | Unelaborated facts 36 33.33 9 10.11 
ideas Elaborated facts 25 23.15 40 44.94 
Unelaborated explanations 10 9.26 17 19.10 
Elaborated explanations 14 12.96 13 14.61 
neal knowledge Creating shared 29 26.85 8 8.99 
uilding understanding 
Negotiating a fit 40 37.04 48 53.93 
Problem-centred idea uptake 20 18.52 23 25.84 
Synthesizing notes 9 8.33 9 10.11 
Scientificness (Mean) 2.95 3.25 
Epidemic complexity 2.02 2.43 


(Mean) 


4. DISCSSION AND CONCLUSION 


This study investigated whether sixth graders could collaboratively develop a knowledge-building discourse. 
We primarily analyzed the students’ online discourse on Knowledge Forum, using multifaceted methods such 
as socio-semantic network analysis and content analysis. Socio-semantic network analysis suggested that 
students engaged in productive collaboration and made democratic contributions. They could advance their 
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community’s ideas collectively by generating explanatory-oriented discourse, negotiating a fit between 
diverse ideas, focusing on problem-centered idea uptake and synthesizing ideas. 

This study has contributed to the literature in two ways. First, it showed that elementary students in an 
examination culture could identify and evaluate promising ideas and carry out metacognitive reflection. 
These skills are critical for productive knowledge building, and can help students develop agency and 
self-direction in their ongoing knowledge building work within a supportive learning environment. In prior 
research, many teachers and scholars have expressed doubt that Chinese elementary students could do such 
higher-order work (van Aalst & Chan, 2007). Second, the pedagogical design (including the three 
components of collaborative ethos, staging principle-based tasks for collaboration and reflection, and 
promisingness judgments supported by the Promising Idea Tool in the community context) was conducive to 
elementary-school students. Thus, it could have important implications for the design of technology-rich 
environments to support learners. 

This study had some limitations. One important limitation was that the study focused on discourse, and 
did not investigate changes in the domain knowledge of individual students. The findings provided evidence 
of idea improvement within the discourse, but it is unclear how widespread these changes were among the 
participants, or whether the changes were transferable. Another limitation was that we did not include or 
analyze classroom data sources such as classroom videos and observations. Nonetheless, such data is critical 
for a better understanding of students’ online knowledge-building discourse and what contributes to their 
productive knowledge building. We are now conducting an analysis of classroom data sources to understand 
classroom processes and dynamics conducive to productive knowledge building. 
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